Audio aided navigation

ABSTRACT

An example operation may include a method comprising one or more of recording a first audio sample at a set of waypoints, traveling, by a device, down a route, reaching a first waypoint, notifying the device to record a second audio sample when the device is not at a destination, comparing the second audio sample with each first audio sample, notifying the device to return to a previous waypoint when the comparison does not match, and notifying the device to continue when the comparison does match.

TECHNICAL FIELD

This application generally relates to navigation, and more particularlyto providing audio aided navigation.

BACKGROUND

While it is possible to track a device on short routes to destinations,the normal tracking on devices such as mobile devices are more accurateover large distances where the device moves at a greater distance thanshort routes, such as inside a building. Therefore, it is necessary tooffer an application that verifies that a device, and a personassociated with the device, is on a correct route to a destination viaother methods, such as comparisons of audio recorded by the device atthe current location and previously recorded audio at previouslydetermined locations.

SUMMARY

An example operation may include a method comprising one or more ofrecording a first audio sample at a set of waypoints, traveling, by adevice, down a route, reaching a first waypoint, notifying the device torecord a second audio sample when the device is not at a destination,comparing the second audio sample with each first audio sample,notifying the device to return to a previous waypoint when thecomparison does not match, and notifying the device to continue when thecomparison does match.

Another example operation may include a system comprising a devicecontaining a processor and memory, wherein the processor is configuredto perform one or more of record a first audio sample at a set ofwaypoints, travel, by the device, down a route, reach a first waypoint,notify the device to record a second audio sample when the device is notat a destination, compare the second audio sample with each first audiosample, notify the device to return to a previous waypoint when thecomparison does not match, and notify the device to continue when thecomparison does match.

A further example operation may include a non-transitory computerreadable medium comprising instructions, that when read by a processor,cause the processor to perform one or more of recording a first audiosample at a set of waypoints, traveling, by a device, down a route,reaching a first waypoint, notifying the device to record a second audiosample when the device is not at a destination, comparing the secondaudio sample with each first audio sample, notifying the device toreturn to a previous waypoint when the comparison does not match, andnotifying the device to continue when the comparison does match.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of one embodiment of the current application.

FIG. 2A is a block diagram illustrating a computer system of oneembodiment of the current application.

FIG. 2B is a table displaying entries in an audio table in oneembodiment of the current application.

FIG. 2C is a table of audio files with deltas in one embodiment of thecurrent application.

FIG. 3A is a GUI representation of a notification in one embodiment ofthe current application.

FIG. 3B is another GUI representation of a notification in oneembodiment of the current application.

FIG. 3C is yet another GUI representation of a notification in oneembodiment of the current application.

FIG. 4A is a map of an office environment in one embodiment of thecurrent application.

FIG. 4B is another map of an office environment in one embodiment of thecurrent application.

FIG. 5 is a flowchart depicting taking audio samples in one embodimentof the current application.

FIG. 6 is a message flow of one embodiment of the current application.

FIG. 7 is a system diagram of one embodiment of the current application.

FIG. 8 is a diagram of a motion speaker in one embodiment of the currentapplication.

FIG. 9 is a diagram of a transport seat with speakers in one embodimentof the current application.

FIG. 10 is a diagram of occupants in a transport in one embodiment ofthe current application.

FIG. 11 is another message flow in one embodiment of the currentapplication.

DETAILED DESCRIPTION

It will be readily understood that the instant components and/or steps,as generally described and illustrated in the figures herein, may bearranged and designed in a wide variety of different configurations.Thus, the following detailed description of the embodiments of at leastone of a method, system, component and non-transitory computer readablemedium, as represented in the attached figures, is not intended to limitthe scope of the application as claimed but is merely representative ofselected embodiments.

The instant features, structures, or characteristics as describedthroughout this specification may be combined in any suitable manner inone or more embodiments. For example, the usage of the phrases “exampleembodiments”, “some embodiments”, or other similar language, throughoutthis specification refers to the fact that a particular feature,structure, or characteristic described in connection with the embodimentmay be included in at least one embodiment. Thus, appearances of thephrases “example embodiments”, “in some embodiments”, “in otherembodiments”, or other similar language, throughout this specificationdo not necessarily all refer to the same group of embodiments, and thedescribed features, structures, or characteristics may be combined inany suitable manner in one or more embodiments.

In addition, while the term “message” may have been used in thedescription of embodiments, the application may be applied to many typesof network data, such as, packet, frame, datagram, etc. The term“message” also includes packet, frame, datagram, and any equivalentsthereof. Furthermore, while certain types of messages and signaling maybe depicted in exemplary embodiments they are not limited to a certaintype of message, and the application is not limited to a certain type ofsignaling.

As a person navigates to a destination, there will be certain audiopatterns emanating from various locations during the route. Some ofthese audio patterns should be consistently similar according to a pathtaken to a specific destination.

As a person is walking along a path to a destination, there is a certainnoise pattern that is usually present. For example, if a user is walkingthrough an office environment, noises will be present that may be usedas cues to verify if the path to the desired destination is beingfollowed. These noises may be unnoticeable to a person if they are notattempting to sense them, but various locations may have a consistentnoise stamp that is recorded and can be compared to audio samples thatare taken.

As a person is arriving at a location, or passing a location, such as abreak room, for example, the noise inside the break room will begin torise and lessen as the person leaves the break room area. Noises areusually emitted from devices that are usually unnoticed by people yetare there none-the-less. The noises may be from a refrigerator, anicemaker, or other appliances in the break room or from individuals inthe break room at certain times.

Other noises may be present in particular areas of an environment. Forexample, from air conditioner vents, outside equipment, the hum offlorescent lights, etc. These noise patterns may be picked up by devicesrecording audio at particular coordinates.

Also, in various locations (such as cubes where people work) also emitparticular noises. For example, in a row of 10 cubes along a path, anoise pattern is possible to detect according to whether or not peopleare working in the cubes or not.

In one embodiment, a device that is recording audio normalizes the audiopatterns recorded along a particular path (for example, a path ofcubes). Audio is recorded at areas within the cubes on the path and theaudio samples are normalized. The client device 102 recording the audiomay send the audio samples to a server 106 wherein the audio samples arenormalized.

The averaging out process may average out recorded audio taken atvarious coordinates wherein various sounds are used as a validation ofan audio sample match. For example, the averaging out process takesrecorded audio at various coordinates along the path of cubes whereinthe noises of people working in a cube are used as a validation of anaudio sample match.

The averaging out is functionally implemented by recording via thecurrent application executing on the device 102, audio during differentparts of the day along a path of cubes, for example 7 cubes (henceforthreferred to as the “multiple path samples”). The audio is notnormalized. Normalization of audio is utilized for two reasons:

-   -   To obtain the maximum volume of the stream without changing its        dynamic range.    -   To match the volumes in the audio of different portions within        the audio stream.

In this situation, it is desirable to obtain the possible audio alongthe path of cubes without knowing beforehand if the cubes are empty,partially occupied, or all occupied. Therefore, the multiple pathsamples are analyzed such that a number of samples along the path, forexample 2 samples with the highest amount of audio are utilized for thefinal audio sample along the path.

Therefore, if anyone is working in any of the cubes, the sample may morelikely match the audio sample taken anywhere along the path.

Therefore, if for example no one is working in the first 3 cubes, but aperson is working in the 4^(th) and 5^(th) cube, the audio sample takenalong the path of cubes may match because the audio samples along thepath were averaged out such that a person working in any of the cubeswould match the recorded audio, even though no one was working in thefirst 3 cubes.

In addition, if no one is working in the first 3 cubes, but a person isworking in the 4^(th) and 5^(th) cube, the audio sample taken along thepath of cubes may match the audio sample(s) known to emanate from the4^(th) cube and the 5^(th) cube, such that a person walking along thepath with a device capable of receiving and/or processing audio, couldreceive validation (or not) that they were on the correct path.

The current process of obtaining audio along the path of the cubes (7cubes for example) will record the audio at cube 3 and obtain noises atthat location coming from cubes 4 and 5 even though no one is in cubes1, 2 and 3. This process continues as the recording device moves closerto the first occupied cube, cube 4.

In another embodiment, the final audio used compares the stored (forexample at server 106) audio from different audio samples. It comparesthe received audio against the audio stored:

-   -   wherein no one was in the first cube    -   wherein no one was in the first two cubes    -   wherein no one was in the first three cubes    -   wherein no one was in the first three cubes, but cube 4 was        occupied    -   etc.

For example, noise or sound can be picked up from the low hum ofmachines such as printers, refrigerators, computers, etc. All of thesemachines, which are generally static in nature, emit sounds mostly fromfans utilized to cool their motors.

One embodiment of the current application seeks to compare audio asreceived by a device on the user to previously recorded noises tovalidate the current path of the user.

Referring to FIG. 1, illustrates a block diagram of one embodiment ofthe current application 100 in accordance with the present disclosure.

The system includes a client device 102. A client device may be at leastone of a mobile device, a tablet, a laptop device, and/or a personaldesktop computer. The client device is communicably coupled to thenetwork 104. It should be noted that other types of devices might beused with the present application. For example, a PDA, an MP3 player, orany other wireless device, a gaming device (such as a hand held systemor home based system), any computer wearable device, and the like(including a P.C. or other wired device) that may transmit and receiveinformation may be used with the present application. The client devicemay execute a user browser used to interface with the network 104, anemail application used to send and receive emails, a text applicationused to send and receive text messages, and many other types ofapplications. Communication may occur between the client device and thenetwork 104 via applications executing on said device and may beapplications downloaded via an application store or may reside on theclient device by default. Additionally, communication may occur on theclient device wherein the client device's operating system performs thelogic to communicate without the use of either an inherent or downloadedapplication.

The system 100 includes a network 104 (e.g., the Internet or Wide AreaNetwork (WAN)). The network may be the Internet or any other suitablenetwork for the transmitting of data from a source to a destination.

A server 106 exists in the system 100, communicably coupled to thenetwork 104, and may be implemented as multiple instances wherein themultiple instances may be joined redundant network or may be singular innature. Furthermore, the server may be connected to database 108 whereintables in the database are utilized to contain the elements of thestored data in the current application, such as Structured QueryLanguage (SQL), for example. The database may reside remotely to theserver coupled to the network 104 and may be redundant in nature.

Referring to FIG. 2A, a block diagram illustrating a computer system 200upon which embodiments of the current invention may be implemented, forexample. The computer system 200 may include a bus 206 or othercommunication mechanism for communicating information, and a hardwareprocessor 205 coupled with bus 206 for processing information. Hardwareprocessor 205 may be, for example, a general purpose microprocessor, forexample.

Computer system 200 may also include main memory 208, such as a randomaccess memory (RAM) or other dynamic storage device, coupled to bus 206for storing information and instructions to be executed by a processor205. Main memory 208 also may be used for storing temporary variables orother intermediate information during the execution of instructions tobe executed by a processor 205. Such instructions, when stored in thenon-transitory storage media accessible to processor 205, may rendercomputer system 200 into a special-purpose machine that is customized toperform the operations specified in the previously stored instructions.

Computer system 200 may also include a read only memory (ROM) 207 orother static storage device which is coupled to bus 206 for storingstatic information and instructions for processor 205. A storage device209, such as a magnetic disk or optical disk, may be provided andcoupled to bus 206 which stores information and instructions.

Computer system 200 may also be coupled via bus 206 to a display 212,such as a cathode ray tube (CRT), a light-emitting diode (LED), etc. fordisplaying information to a computer user. An input device 211 such as akeyboard, including alphanumeric and other keys, is coupled to bus 206,which communicates information and command selections to processor 205.Other type of user input devices may be present including cursor control210, such as a mouse, a trackball, or cursor direction keys whichcommunicates direction information and command selections to processor205 and controlling cursor movement on display 212.

According to one embodiment, the techniques herein are performed bycomputer system 200 in response to a processor 205 executing one or moresequences of one or more instructions which may be contained in mainmemory 208. These instructions may be read into main memory 208 fromanother storage medium, such as storage device 209. Execution of thesequences of instructions contained in main memory 208 may causeprocessor 205 to perform the process steps described herein. Inalternative embodiments, hard-wired circuitry or embedded technology maybe used in place of or in combination with software instructions.

The term “storage media” as used herein refers to any non-transitorymedia that may store data and/or instructions causing a machine tooperation in a specific fashion. These storage media may comprisenon-volatile media and/or volatile media. Non-volatile media mayinclude, for example, optical or magnetic disks, such as storage device209. Volatile media may include dynamic memory, such as main memory 208.Common forms of storage media include, for example, a hard disk, solidstate drive, magnetic tape, or other magnetic data storage medium, aCD-ROM, any other optical data storage medium, any physical medium withpatterns of holes, a RAM, a PROM, and EPROM, a FLASH-EPROM, NVRAM, anyother memory chip or cartridge.

Various forms of media may be involved in the carrying one or moresequences of one or more of the instructions to processor 205 forexecution. For example, the instructions may initially be carried on amagnetic disk or solid state drive of a remote computer. The remotecomputer may load the instructions into its dynamic memory and send theinstructions over a medium such as the Internet 202.

Computer system 200 may also include a communication interface 204coupled to bus 206. The communication interface may provide two-way datacommunication coupling to a network link, which is connected to a localnetwork 201.

A network link typically provides data communication through one or morenetworks to other data devices. For example, the network link mayprovide a connection through local network 201 to data equipmentoperated by an Internet Service Provider (ISP) 202. ISP 202 providesdata communication services through the world wide packet datacommunication network now commonly referred to as the “Internet” 202.Local network 201 and Internet 202 both use electrical, electromagnetic,or optical signals that carry digital data streams. The signals throughthe various networks and the signals on network link and throughcommunication interface 204, carrying the digital data to and fromcomputer system 200, are example forms of transmission media.

Computer system 200 can send messages and receive data, includingprogram code, through the network(s) 202, the network link, and thecommunication interface 204. In the Internet example, a server 203 maytransmit a requested code for an application program through Internet202, local network 201 and communication interface 204.

The received code can be executed by processor 205 as it is received,and/or stored in storage device 209, or other non-volatile storage forexecution at a later time.

Every action or step described herein is fully and/or partiallyperformed by at least one of any element depicted and/or describedherein. Additionally, any step depicted may be performed in any order,other than presented.

In the following description, for the purposes of explanation, numerousspecific details are set forth in order to provide a thoroughunderstanding of the present disclosure. It will be apparent, however,that the present disclosure may be practiced without these specificdetails. In other instances, well-known structures and devices are shownin block diagram form in order to avoid unnecessarily obscuring thepresent disclosure. Modifiers such as “first”, “second”, and “third” maybe used to differentiate elements, but the modifiers do not necessarilyindicate any particular order. For example, a first party may be sonamed although, in reality, it may be a second, third, and/or fourthparty.

US patent application 2016/0314209 entitled “Navigating with A CameraDevice”, henceforth referred to as the '209 patent, offers a system andmethod that allows the use of a camera or camcorder on a device to aidin a navigation system. As the device geographically moves from astarting location to a destination, images and/or video taken by adevice associated with the user are analyzed wherein it is determinedwhether or not the device is on a valid path to the destination.

The movement of the user may be determined in a variety of ways in the'209 application, including the device being tracked using at least oneof GPS technology, and indoor positioning system (e.g. WI-FI), anaccelerometer on the device, or an altimeter on the device.

Images taken from the device in the '209 application are stored andcompared to previously taken images along the path of the device from astarting location to the destination. In some embodiments, the imagedata is sent to a navigation module, which may search through storedimages of locations to find an image sufficiently similar to the imageobtained by the device.

The current application seeks to offer a solution that utilizes dataother than image data (e.g. audio) to ascertain the current location ofthe device and determine if the device is properly routing to adestination.

To record audio from a device, the software initiates recordingfunctionality. For example, in the Android operating system, thefollowing Java code is utilized to initiate the recording of audio onthe device:

md = new MediaRecorder( );md.setAudioSource(MediaRecorder.AudioSource.MIC);md.setOutputFormat(MediaRecorder.OutputFormat.MPEG_4);md.setOutputFile(recordFile);md.setAudioEncoder(MediaRecorder.AudioEncoder.AAC); md.prepare( );md.start( );

In one embodiment, the recorded media is stored in the device 102locally then sent to a server 104 for processing. In another embodiment,the recorded media is stored in the device 102.

Audio is recorded and stored in an environment. A recording device isused to record sounds in the environment wherein the audio is storedwith the precise geographical location of the device. The data is thenstored in a database, such as database 108.

Referring to FIG. 2B, showing entries in an audio table in oneimplementation of the current application. The table stores a sampleaudio recording, along with the exact geographic location of where theaudio was taken.

A UID column is present as the first column and may be used to index therow, in some implementations. A second and third column provides theexact geographic location of where the audio is taken. A fourth andfinal column is a link to the audio file, or the actual audio file. Inmany database systems, the actual file is stored in a server's filesystem, and the database simply points to the actual file.

Each audio file is sample audio taken at the corresponding geographiclocation. The audio file is not long in length but is long enough toprovide an example of the audio condition at the location, for example 5seconds.

In one embodiment, audio samples are taken on a device, such as device102 wherein an audio sample is recorded on the device at particularintervals. The device begins at a starting point and moves to thedestination. This path may be provided via a mapping application whereinthe starting location and the destination location is determined. As thedevice moves to different points along the shortest or best path to thedestination, audio recordings are made.

The recording of the path begins via the execution of the currentapplication executing on the device 102. Along the path, a notificationis made on the device wherein the user is notified to stop while audiois recorded. This notification may be an alert sound on the device, anotification message on the GUI of the current application executing onthe device, or a combination of both. The recording of the audio at eachinterval is recorded for a predetermined amount of time, for example 5seconds. When the audio sample is complete, another notification ispresented on the device wherein the user then moves along the path untilthe next notification is presented.

The distance between notifications alerting the user that an audiosample is needed is determined by the distance between the startinglocation and the destination location wherein the delta distance issplit by a predetermined value that changes based on the amount ofdistance between the starting location and the destination location.

For example, if the distance is 1000 yards, then audio samples are takenevery 30 yards such that there are a total of 33 audio samples taken, ora sample taken every 33 yards. As another example, if the distance is180 yards, then audio samples are taken every 9 yards such that thereare a total of 20 samples taken, or a sample taken every 20 yards.

The exact number of samples required for a given distance isprogrammatically determined and one versed in common program techniquesmay easily determine the best number of samples over a given routelength without deviating from the scope of the current invention.

Taking an audio sample along a route to a destination requiresadditional time. Each audio sample requires the user to stop and take anaudio sample, which can take up to 5 seconds, for example, then theobtained audio sample taken at the device 102 must be compare againstthe audio sample store in the server 106. The requirement of takingsamples at intervals that are too close together may allow the currentapplication to be not useful, therefore it is important to require thatsamples be taken at intervals wherein the path to the destination isassured, yet not bother the user with having to take audio samples toooften.

As an example, setting points along a route to a destination (otherwiseknown in the art as waypoints) allows for actions to occur at each pointreached along the destination. The following code utilizes a mapping APIto set two waypoints along a route:

“routes”: [   {     “bounds” : {      “one” : {        “lat” :37.8079996,        “lng” : −122.4074334     },      “two” : {       “lat” : 37.7881005,        “lng” : −122.4203553      }     },  }]

When a waypoint is reached on a destination, a notification is presentedon the GUI of the current application to request another audio sample.

Due to the difficulty of determining the exact location of the deviceinside a building, the approximate location of the device is used as thewaypoint. For example, if the device is within a radius of 10 yards fromthe set waypoint, an audio sample is requested. The server 106calculates the waypoint with a radius of equal to the set radius (e.g.10 yards) and uses the radius coordinates to indicate that the devicehas arrived at the waypoint.

In another embodiment, the waypoints are not used along the route to thedestination. The current geographic location of the device 102 iscompared against the previous geographic location when a sample wastaken, and the delta between the two points are used wherein the deltamust be equal or greater to a previously determined distance. Thepreviously determined distance is a distance hardcoded in the softwareof the current application, for example 10 yards.

Audio samples are compared against stored audio files taken at the samegeographic location to determine similarities. The process of analyzingaudio files is called acoustic fingerprinting. Acoustic fingerprintingis a condensed, digital summary, deterministically generated from anaudio signal that can be used to identify an audio sample or quicklylocate similar items in an audio database.

A robust acoustic fingerprint algorithm takes into account theperceptual characteristics of the audio. If two files sound alike to thehuman ear, their acoustic fingerprints should match, even if theirbinary representations are quite different. Acoustic fingerprints arenot bitwise fingerprints, which must be sensitive to any small changesin the data. Acoustic fingerprints are more analogous to humanfingerprints where small variations that are insignificant to thefeatures the fingerprint uses are tolerated. A smeared human fingerprintimpression can accurately be matched to another fingerprint sample in areference database; acoustic fingerprints work in a similar way.

Perceptual characteristics often exploited by audio fingerprints includeaverage zero crossing rate, estimated tempo, average spectrum, spectralflatness, prominent tones across a set of frequency bands, andbandwidth.

Most audio compression techniques (AAC, MP3, WMA, Vorbis) will makeradical changes to the binary encoding of an audio file, withoutradically affecting the way it is perceived by the human ear. A robustacoustic fingerprint will allow a recording to be identified after ithas gone through such compression, even if the audio quality has beenreduced significantly. For use in radio broadcast monitoring, acousticfingerprints should also be insensitive to analog transmissionartifacts.

There are tools available that compare two acoustic files forsimilarities. For example, AcoustID ( ) is on open sourced applicationthat offers the advantage of being actively developed. Many acousticfingerprint applications are utilized in today's market for musicrecognition, but these offerings may be used to determine similaritiesin audio files.

As an example, a Java class called fingerprintSimilarity may be used toobtain a score of the comparison of two audio files. The following codecompares the audio samples of two way files:

Wave wave1 = new Wave(audioFilePathNew); Wave wave2 = newWave(audioFilePathStored); FingerprintSimilarity fingerprintSimilarity =    wave1 .getFingerprintSimilarity(wave2); float score =fingerprintSimilarity.getScore( ); float similarity =fingerprintSimilarity.getSimilarity( );Log.d(“Similarsound”,     “Score: ” + score + “\n Similarity : ”+ similarity);

In another embodiment, the expected audio is stored along with thelatitude/longitude of where the audio is recorded, then a delta betweenthe audio stored and the audio received at the same geographic location.The delta is determined via logic in the code, for example utilizing thefunctionality of audio fingerprinting and/or spectral analysis or thelike. For example, the following table contains the two audio files, anda score of the similarities of the two:

Referring to FIG. 2C shows a table of audio files, with deltas. In oneembodiment, the delta score is used to determine the response data tothe client device 102. The higher the delta score (up to 10) the moresimilar the two compared audio files are. If the delta score for ageographic location is above 7, it is assumed that the audio file issimilar and the response to the device is that they are on the route.

If the response is between 3 and 7, it is not a verifiable comparison,the data in the message informs the user to take another recording at anearby location, and another comparison is made with the new file.

If the response is below 2, text in the notification informs the usethat they may be on an incorrect route and to return to the lastverified waypoint and continue along another path.

Referring to FIG. 3A is a GUI representation of a notification of anaudio sample in one implementation of the current application 300. Anotification is presented on the client 102 with a title indicating thepurpose of the message. The message “Please stop and take an audiosample at this location.” informs the user the action to take.

Two buttons at the bottom of the notification allow user to either press“Begin” to initiate the recording of audio at the current location, and“Cancel” to remove the notification window and continue.

In another embodiment, an audio notification is made, such as a beep,informing the user of the client device 102 to begin recording an audiosample. When the audio sample is complete, two beeps are made informingthe user to walk to the next waypoint along the route.

Referring to FIG. 3B, a GUI representation of a notification of an audiosample in one implementation of the current application 310. Anotification is presented on the client 102 with a title indicating thepurpose of the message. The message “You are on the right path to thedestination.” informs the user that the previous audio sample wasvalidated, and the current path is the correct path to the destination.

In another embodiment, other verifiers are used via the currentapplication to assure the client device is on the correct route. Forexample, passing by a location where a particular sound is expected, setat a specific geographical latitude and longitude. This sound may notparticularly be previously recorded at that location, but obtained viaanother source, such as file accessed via the Internet 104, or a similarsource.

In another embodiment, a client device may capture a photo at a specificlatitude/longitude wherein the photo is sent to the server 106 in thesimilar fashion as the previously mentioned audio sample. A set ofphotos are stored at the server tied to a latitude/longitude eitherlocally in the server 106, in a database such as database 108, or storedremotely in the network 104. The received photo is compared against thestored photo at the same or similar geographic location and feedback isprovided therein.

In yet another embodiment, a device containing a sensor and a motiondevice, for example, is set at a geographic location such that a messageis sent to the server 106 upon detecting the client device 102 whereinthe server responds back to the said client device that the path isindeed the correct path.

Referring to FIG. 3C, a GUI representation of a notification of an audiosample in one implementation of the current application 320. Anotification is presented on the client 102 with a title indicating thepurpose of the message. The message “You may be on the wrong path.Return to previous point and continue.” informs the user that thepreviously sent audio sample did not validate against the stored audiosample at that location. The user should return to the previouslyvalidated point and continue.

In another embodiment, when the user returns to the previous, validatedpoint the current application executing on the client device 102 doesnot validate audio as the audio would have previously been validated atthat location. The next notification is triggered at the arrival of thedevice in the next waypoint.

Referring to FIG. 4A, a map of an office environment 400 indicating theroute from a starting point, the determined waypoints, and the finaldestination.

A starting point 402 is determined as well as a final destination 412.There are 3 waypoints along the route 406, 408, and 410. Each waypointhas a radius 406 (e.g. 2 yards) wherein the device is determined to havearrived in that waypoint if the device enters the radius of the waypoint406.

The waypoints 404 and 406 may be set as distances from the previouspoint (e.g. the distance between the starting point 402 and the firstwaypoint 404 is 10 yards, the distance between the second waypoint 406and the previous way point 404 is 10 yards, etc.

In another embodiment, the current application may provide not theshortest distance between the starting location and the ending location,but a route that may be longer in length, but allow for additionalinteractions.

The current application executing on the client device 102 maydetermine, through interactions with the user's (User A) client device'sscheduling application that the user has an upcoming meeting withanother user (User B) of whom is on an alternative route to the finaldestination. This interaction is through an API of the schedulingapplication, for example. Therefore, the user may wish to discuss theagenda of the meeting with User B to avoid a formal meeting as iscurrently scheduled.

Therefore, the current application will map a route in a similar fashionto the many routes a navigation application may offer a driver to afinal destination.

The current application issues a notification to the user's (User A)client device 102 executing the current application. The notificationhas the following text:

-   -   [event.attendee] is on a route to your final destination and you        have an upcoming meeting [event.start Time] about [event.title].        Do you wish to pursue that route? [Yes] [No]

If the response is “Yes”, then the current application executing on theclient device 102 routes the device along the path to User B. If theresponse is “No”, then the route is not altered.

In another embodiment, the alternative route may include tasks such as aroute by a printer if the current application determines that a printouthas been requested via a printer on the route. The current applicationinteracts with the enterprise's printer queue software to determine if aprintout has been requested.

In another embodiment, the current application executing on the device102, tracking the progress of the device on either the route oralternative route, may calculate the time of arrival to the printingdevice such that the document(s) to be printed are queued wherein theprintout will be at the top of the print stack, allowing the user toeasily remove the printout from the printer.

Referring to FIG. 4B, a map of an office environment 420 indicating analternative route from a starting point, the determined waypoints, andthe final destination.

The figure depicts the normal route as indicated via a solid line, andthe alternative route 422. On the alternative route, there exists awaypoint set to a printer 424, and a waypoint set to the location ofUser B 426, as further discussed herein.

Referring to FIG. 5, a flowchart depicting the taking of audio samplesin one implementation of the current application 500.

The route is begun wherein the device begins the route along a path tothe final destination. A waypoint is reached 504. This waypoint may be aset of coordinates as previously determined, or a distance traveledsince the previous point.

A check is made whether the device 102 is at the destination 504. If thedevice is at the final destination, the process ends 506. If thedestination is not reached, a notification is presented on the displayof the client device 102 in the current application executing therein508 to begin recording an audio sample 300.

The sample is completed 510 wherein the audio sample recorded iscompared against the audio sample of the current waypoint 512. A checkis made if the audio samples match 514. If the samples match anotification is presented on the display of the client device 102 in thecurrent application executing therein 516 for the user to continue 310.

If the samples do not match, a notification is presented on the displayof the client device 102 in the current application executing therein510 for the user to return to the previous point 320.

Referring to FIG. 6, a message flow of a possible implementation of thecurrent application wherein audio samples are sent to a server forprocessing 600.

The client device 102 begins down a path to the final destinationwherein a waypoint is reached 602. The client is notified to take anaudio sample 602. When the sample is complete, an audio sample message606 is sent to the server 106. The audio sample message contains atleast two parameters: the sample of audio (audio1), and the currentcoordinates of the device (coordinates).

The server 106 may query the database 108 to obtain the audio samplepreviously recorded at the given coordinates (coordinates). A responsemessage 610 is returned with the audio sample (audio2) at thosecoordinates 610.

The server 106 compares the two audio samples 612 (audio1 and audio2)wherein it is determined if the received audio sample from the device(audio1) matches the previously stored audio sample at that coordinate(audio2) as further disclosed herein.

A response is sent to the client device 102 with a notification(notification) of the success of the audio comparison 614. Thenotification will either reflect that a match was made when the audiowas compared 310, or a match was not made, and the user should return tothe previous point and continue 320.

The client device 102, upon receiving the response message 614 displaysthe notification 616 and the process continues.

In an alternate embodiment, the stored audio files at predeterminedcoordinates are stored in the server 106 wherein messaging 608 and 610does not occur.

In another embodiment, transports have an entertainment system,henceforth referred to as a “radio”. These radios may contain multiplesources of data:

-   -   AM/FM radio reception    -   Compact disk    -   Integration with other devices such as mobile phones or music        devices    -   Video disks such as Blu-Ray or DVDs

Many transports have multiple speakers throughout the interior such thatregardless of where a person is sitting in the transport, they are ableto hear sounds from the radio.

A problem arises when an occupant moves their head and the volumeremains the same. Regardless of the position of the occupant's head, thevolume remains the same.

The current invention offers a system and methods in a transport whereina transport may be an automobile, airplane, train, bus, boat, or anytype of vehicle that normally transports people from one place toanother. This allows the speaker(s) near an occupant's ears to bemodified either by an automatic change in the volume or by the movementof the speaker(s) according to the position of the occupant's ears.

Referring to FIG. 7, a system diagram of the current application 700. Atleast one device is located in a transport 702, which communicates witha network 708. The network communicates with a server 710.

The transport 702 contains a device such as an in-transport navigationsystem or entertainment system, or any device including a processor andmemory 704, henceforth referred to as the transport system and acts asthe main communication device for the current application, and/or aclient device 706, and an image/video camera 707, which communicateswith the transport system 704. The client device being a device maycommunicate with the transport system 704 or may directly connect withthe network 708. Transport system 704 contains a processor and memory.The processor receives input, analyzes/parses the input, and provides anoutput to one or more systems, modules, and/or devices.

The client device may be least one of a mobile device, a tablet, or alaptop device. It should be noted that other types of devices might beused with the present application. For example, a PDA, an MP3 player, orany other wireless device, a gaming device (such as a hand held systemor home based system), any computer wearable device, and the like(including personal computer or other wired device) that may transmitand receive information may be used with the present application. Theclient device and/or the in-transport navigation system may execute auser browser used to interface with the network 708, an emailapplication used to send and receive emails, a text application used tosend and receive text messages, and many other types of applications.Communication may occur between the client device and/or thein-transport navigation system and the network 708 via applicationsexecuting on said device and may be applications downloaded via anapplication store or may reside on the client device by default.Additionally, communication may occur on the client device wherein theclient device's operating system performs the logic to communicatewithout the use of either an inherent or downloaded application.

A server 710 exists in the system, communicably coupled to the network708, and may be implemented as multiple instances wherein the multipleinstances may be joined to form a complete cryptocurrency wallet or maybe singular in nature. Furthermore, the server may be connected to adatabase (not depicted) wherein tables in the database are utilized tocontain the elements of the system and may be accessed via queries to adatabase, such as Structured Query Language (SQL), for example. Thedatabase may reside remotely to the server coupled to the network 708and may be redundant in nature.

The modification of the volume of headrest speakers, as well as themovement of the speaker is dependent on the initial position andmovement of the occupant's head.

To track the position and movement of the occupant's head, the currentapplication uses a camera 707 mounted such that all occupants aretracked.

In one embodiment, an image and/or video camera (henceforth referred toas a “monitoring camera” is mounted inside the transport 702 such thatall people in the transport may be captured and monitored. For a mountedcamera, an image is taken at a determined interval. The interval ishardcoded in the logic of the transport system 704 and set to adetermined value, such as an image is taken every 1.5 seconds. The imagemay be stored in the transport system 704 of the current applicationeither locally inside the transport, or in a remote server 710 such thatcommunication between the server and the remote server is through thenetwork 708.

In another embodiment, the camera is mounted in the interior roof of thetransport, around the rear-view mirror and points towards the back suchthat all occupants in the transport can be monitored. The tracking ofthe occupants' head is performed via the transport system 704 throughtracking software. For example, the code below utilizes a popular camerain a gaming system to track the movement of a player's head.

namespace SkeletalTracking { public partial class MainWindow : Window {  public MainWindow( )   {    InitializeComponent( );   }   bool closing= false;   const int skeletonCount = 6;   Skeleton[ ] allSkeletons = newSkeleton[skeletonCount];   private void Window_Loaded(object sender,RoutedEventArgs e)   {    kinectSensorChooser1.KinectSensorChanged +=new DependencyPropertyChangedEventHandler(kinectSensor-Chooser1_KinectSensorChanged);   }

The use of this logic is utilized to track the occupants' headmovements, therefore allowing the sending of commands to the speakers tomodify the volume or change direction for each speaker.

The headrest speaker(s) are moveable wherein the controls of themovements are determined by the transport system 704. The transportsystem sends messages to the headrest speaker informing the speaker(s)of the direction of movement and amount of movement. The transportsystem may also send a command to return the speaker to a defaultposition, such as pointing straight.

Referring to FIG. 8, a diagram of a motion speaker. FIG. 8 shows themovement of a headrest speaker 1002,1210 in another implementation ofthe current system 800. There are two axes wherein the speaker 802 isable to move: the X-axis 804 and the Y-axis 806.

In one embodiment, the movement of the headrest speaker tracks theoccupant's 1002, 1004, 1006, 1008 ears. The movement signals arereceived from the monitoring camera 707, which sends images/video to thetransport system 704 for processing. The transport system tracks themovements of the occupants' head in real-time, providing the ability forthe altering of the direction of the headrest speakers as the occupants'head move.

Each seat in the transport has a pair of speakers near the occupant'shead, henceforth referred to as “headrest speakers”. Headrest speakersare on the left and right side of the occupant's head when the occupantis sitting in the seat.

Referring to FIG. 9, a transport seat with speakers 900, in oneembodiment of the current application. The headrest speakers 902 areplaced in the seat behind the left and right ears.

In another embodiment, the headrest speakers are along a track 904wherein they may be moved vertically to accommodate shorter or talleroccupants. A lever 906 placed alongside the side of the seat allows theheadrest speaker to move such that each headrest speaker may be movedinside the seat. The seat lining 904 over the speaker area is made ofmesh such that regardless where the speaker is placed alongside thetrack, it is able to produce full sound due to the construction of themesh covering.

In another embodiment, if the occupant does not occupy the seat, theheadrest speakers 902 turn off.

In another embodiment, the speakers turn on and are ready to receivesignals from the transport system when the occupant enters the transportand sits in the seat with the headrest speakers 902.

Referring to FIG. 10, a depiction of occupants of a transport 1000, inone embodiment of the current application. The figure is the view thatis from the perspective of the monitoring camera 707, in oneimplementation of the current application. In one implementation of thecurrent application, the monitoring camera is mounted in the interiorroof of the transport and is able to view the heads of all of theoccupants in the transport.

There are four occupants in the transport 1002, 1004, 1006, and 1008 inthe figure. The driver may be in the driver's seat 1004 (outside Britishcolonies) in some implementations. All occupants are sitting down inseats. There are two speakers of the current application 1010 on bothsides of all of the occupants' heads.

The monitoring camera 707 receives video that is sent to the transportsystem 704 where the image is analyzed as further disclosed herein. Thetransport system then, according to the determined motion of theoccupant(s), will raise/lower the volume of the headrest speaker(s)and/or issue commands to change the direction of the headrestspeaker(s).

In one embodiment, the headrest speakers 1010 automatically adjust forhead movements. Normally, as a person brings the ear closer to thespeaker, the volume appears to increase. In the current application,this phenomenon is accounted for. For example, as an occupant turns tothe left, the left headrest speaker is lowered, and the right headrestspeaker is raised, thus giving the impression that the volume of theaudio has not been altered. In this scenario, the location of the leftear is closer to the headrest speaker and the position of the right earis further from the headrest speaker.

In another embodiment, a headrest speakers 1010 may be turned off or thevolume of the speaker set to 0 (zero) at times when the position of anear is a particular distance from the headrest speaker.

The volume of each headrest speaker 1010 is independently controlled viathe current system. Furthermore, the system tracks the occupant's headat all times such that each headrest speaker may adjust according to theposition of the occupant's head. Therefore, if the occupant moves downin the seat, the speakers point down. If the occupant moves to the leftor right, the speakers move direction left or right respectfully.

In another embodiment, depending on position of the occupant's head, theheadrest speakers are modified, as is the entire volume of the system.

In another embodiment, each of the speaker's volume in the transport ismodified by the transport system 704 except a separate sub-speaker, asthe volume of the sub-speaker does not easily alter the ability foroccupants to carry on a conversation.

In another embodiment, the headrest speaker 1010 or speaker housing orother device emits a low-grade laser, echo, or signal to detect theobject in front of the speaker. This technology is similar to the sonarfunctionality wherein data is sent out from the headrest speaker and theamount of echo returned is utilized to determine how far an object isfrom the headrest speaker. The data collected by the headrest speakerutilizing the sonar technology is sent to the transport system 704 todetermine any modification of either volume or movement of the headrestspeaker.

Referring to FIG. 11, a flow of some of the embodiments of the currentapplication 1100. There are at least three elements of the overallsystem, a monitoring camera 707, a transport system 704, and headrestspeakers 1010.

In some embodiments, the monitoring camera 707 is replaced by anothermethod of monitoring the occupants' movements. For example, a motiondetection device placed inside the transport or any other device that isused to track and record motion.

The transport system 704 may be part of the computer coupled with thetransport or may exist entirely or partially in a client device 706 suchas a mobile device, or any other device containing a processor andmemory.

Data is received from the monitoring camera 1102, which may be imagesand/or video. This data is sent to the transport system 1104.

The transport system 704 receives the data 1106, which may be storedlocally or in a remote database. The data is analyzed 1108 to examine todetermine (further explained herein), among other functions:

-   -   any head movements of occupants in the transport    -   the removal or addition of occupants in the transport

The headrest speakers 1010 are modified according to the analysis of thedata 1110. This may result in the modification of the volume of theheadrest speakers 1112 and/or the modification of the direction of theheadrest speakers 1114.

What is claimed is:
 1. A method, comprising: recording a first audiosample at a set of waypoints; traveling, by a device, down a route;reaching a first waypoint; notifying the device to record a second audiosample when the device is not at a destination; comparing the secondaudio sample with each first audio sample; notifying the device toreturn to a previous waypoint when the comparison does not match; andnotifying the device to continue when the comparison does match.
 2. Themethod of claim 1, wherein the first waypoint is a one or more of: apreviously determined set of coordinates; and a distance traveled sincea previous point.
 3. The method of claim 1, comprising one or more ofsending to a server the first audio sample; and sending to the serverthe second audio sample.
 4. The method of claim 3, wherein the comparingis performed by the server.
 5. The method of claim 1, comprising:storing one or more of: a first coordinate of the device when recordingof the first audio sample; and a second coordinate of the device whenrecording the second audio sample.
 6. The method of claim 5, wherein inresponse to the comparing, comparing one or more of: the firstcoordinate; and the second coordinate.
 7. The method of claim 6,comprising storing in the database one or more of: the first audiosample; the second audio sample; the first coordinate; and the secondcoordinate.
 8. A system, comprising: a device containing a processor andmemory, wherein the processor is configured to perform: record a firstaudio sample at a set of waypoints; travel, by the device, down a route;reach a first waypoint; notify the device to record a second audiosample when the device is not at a destination; compare the second audiosample with each first audio sample; notify the device to return to aprevious waypoint when the comparison does not match; and notify thedevice to continue when the comparison does match.
 9. The method ofclaim 1, wherein the first waypoint is a one or more of: a previouslydetermined set of coordinates; and a distance traveled since a previouspoint.
 10. The method of claim 1, comprising one or more of send to aserver the first audio sample; and send to the server the second audiosample.
 11. The method of claim 3, wherein the compare is performed bythe server.
 12. The method of claim 1, comprising: store one or more of:a first coordinate of the device when the first audio sample isrecorded; and a second coordinate of the device when the second audiosample is recorded.
 13. The method of claim 5, wherein in response tothe compare, compare one or more of: the first coordinate; and thesecond coordinate.
 14. The method of claim 6, comprising store in thedatabase one or more of: the first audio sample; the second audiosample; the first coordinate; and the second coordinate.
 15. Anon-transitory computer readable medium comprising instructions, thatwhen read by a processor, cause the processor to perform: recording afirst audio sample at a set of waypoints; traveling, by a device, down aroute; reaching a first waypoint; notifying the device to record asecond audio sample when the device is not at a destination; comparingthe second audio sample with each first audio sample; notifying thedevice to return to a previous waypoint when the comparison does notmatch; and notifying the device to continue when the comparison doesmatch.
 16. The non-transitory computer readable medium of claim 15,wherein the first waypoint is a one or more of: a previously determinedset of coordinates; and a distance traveled since a previous point. 17.The non-transitory computer readable medium of claim 15, comprising oneor more of sending to a server the first audio sample; and sending tothe server the second audio sample.
 18. The non-transitory computerreadable medium of claim 17, wherein the comparing is performed by theserver.
 19. The non-transitory computer readable medium of claim 15,comprising: storing one or more of: a first coordinate of the devicewhen recording of the first audio sample; and a second coordinate of thedevice when recording the second audio sample.
 20. The non-transitorycomputer readable medium of claim 19, wherein in response to thecomparing, comparing one or more of: the first coordinate; and thesecond coordinate.